Transactions are at the heart of e-business. Without fast, efficient transactions, orders dwindle and profits diminish. Today's e-business technology, for example, is providing businesses of all types with the ability to redefine transactions. There is a need, though, to optimize transaction performance and this requires the monitoring, careful analysis and management of transactions and other system performance metrics that may affect e-business.
Due to the complexity of modern e-business systems, it may be necessary to monitor thousands of performance metrics, ranging from relatively high-level metrics, such as transaction response time, throughput and availability, to low-level metrics, such as the amount of physical memory in use on each computer on a network, the amount of disk space available, or the number of threads executing on each processor on each computer. Metrics relating to the operation of database systems, operating systems, physical hardware, network performance, etc., all must be monitored, across networks that may include many computers, each executing numerous processes, so that problems can be detected and corrected when (or preferably before) they arise.
Due to the complexity of the problem and the number of metrics involved, it is useful to be able to quickly view information relating to one or more metrics across a period of time. In particular, viewing information on the frequency distribution of data may be useful. Such data distribution information generally may be viewed as a histogram, and many systems that perform statistical analysis are able to display histograms.
Unfortunately, there are a number of drawbacks to using conventional histograms to view time series data, such as the complex metrics discussed above. First, there is often a need to display multiple histograms on a screen when working with time series data, such as metrics. Each histogram typically requires a large amount of screen space, limiting the number of histograms that can be displayed at the same time.
Additionally, histograms are not always useful for discovering trends in data over time, since it may be difficult to see long-term trends in data by viewing a set of standard histograms side-by-side or stacked vertically. Some systems attempt to solve this problem by making the histograms small, and turning them on their side. Unfortunately, even when these steps are taken, it may be difficult to display more than five or six histograms on a single display. It would be impractical in such systems to display fifty or a hundred such histograms in a single display.
Another difficulty with using histograms to view data distribution information over a long period of time is the storage of histogram data. Typically, a large number of individual data samples are needed to construct a histogram. To display histograms for data over a long time period, there are two options available. First, a system can pre-compute the histogram for each required time interval, and save the histogram data. This approach requires many computations and storage of data that may never be used. Alternatively, a system can save all the individual data points over a long time period, so that histograms can be computed as they are needed. This approach requires a large amount of storage, and may require a large memory footprint and a large amount of computation when the histograms are generated. As a result, this approach may not be practical for long periods of time and large numbers of metrics.
Additionally, histograms are somewhat inflexible. For example, they cannot be effectively averaged or merged to condense the display of several time intervals into a single interval. Similarly, they cannot be effectively averaged or merged from multiple copies of the same metric collected from distinct but similar systems. Such data distribution information may be useful for viewing the health and status of an entire system using only a few displays or screens.